Evaluating perceptual and semantic interpretability of saliency methods: A case study of melanoma

In order to be useful, XAI explanations have faithful the AI system they seek elucidate and also interpretable people that engage with them. There exist multiple algorithmic methods for assessing faithfulness, but this is not so interpretability, which typically only assessed through expensive user studies. Here we propose two complementary metrics algorithmically evaluate interpretability of saliency map explanations. One metric assesses perceptual by quantifying visual coherence map. The second semantic capturing degree overlap between textbook features—features human experts use make a classification. We melanoma dataset deep-neural network classifier as case-study explore how our relate each other faithfulness metric. Across six commonly used methods, find none achieves high scores across all three test images, different perform well in regions data distribution. This variation can leveraged consistently achieve using inform mask selection on case-by-case basis. Our provide new way saliency-based allow adaptive combination explanation methods.